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ABSTRACT 


This  thesis  is  a  case  study  that  compares  actual 
costs  to  estimated  costs  for  a  State-of - the-Ar t  ( SOA  ) 
extension.  The  Advanced  Nuclear  Gamma-Ray  Spectrometer 
(ANGAS)  program  initiated  by  the  Defense  Advanced 
Research  Project  Office  ( DARPA )  in  conjunction  with  the 
Research  and  Development  Division  of  Lockheed  Missiles 
and  Space  Company  ( LMSC ) ,  Inc.  is  the  subject  of  this 
case  study. 

This  thesis  identifies:  the  original  description  of 
the  technology  extension;  the  methods  used  at  Lockheed 
to  develop  SOA  extension  cost  estimates;  specific 
factors  that  helped  and  hindered  accurate  cost 
estimating  and  significant  variances  in  cost  and 
technological  progress. 
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I .  INTRODUCTION 
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A.  BACKGROUND 

The  development  of  new  weapon  systems  for  the 
Department  of  Defense  (DOD)  frequently  requires  State 
of-the-Art  ( SOA )  extensions.  The  difficulty  of 
estimating  development  and  early  production  costs  for 
new  systems  when  SOA  extensions  are  involved  often 
leads  to  cost  overruns.  In  fact,  contractors’  costs 
exceed  project  manager’s  estimates  for  new  projects  60 
percent  of  the  time.  [REF.  1  :  p.  50-51.] 

Prior  research  has  not  solved  the  problem  of 
accurately  estimating  SOA  extension  costs.  These 
development  and  production  cost  overruns  make  it 
difficult  for  DOD  to  accurately  compare  existing  weapon 
technology  with  undeveloped  technology. 

Professor  Willis  R.  Greer,  Jr.,  Naval  Postgraduate 
School,  Monterey,  California,  in  conjunction  with  the 
Naval  Sea  Systems  Command  (NAVSEA),  is  attempting  to 
develop  more  reliable  methodology  for  estimating  the 
cost  of  extensions  as  applied  to  electronic  systems. 
This  thesis  provides  data  to  support  that  effort 
through  the  case  study  of  an  SOA  extension. 

B.  OBJECTIVES 

This  thesis  details  the  Advanced  Nuclear  Gamma-Ray 
Spectrometer  (ANGAS)  experiment  program  initiated  by 
the  Defense  Advanced  Research  Projects  Office  in 
conjunction  with  the  Research  and  Development  Division 
of  Lockheed  Missiles  and  Space  Company,  Inc.  The 
objectives  are  outlined  below: 

1.  Identify  the  original  description  of  the 
technology  extension, 

2.  Document  estimated  development  costs. 
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3. 


I 

I 
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Describe  the  methods  used 
estimates , 

4.  Compare  actual  cost  data  t 

5 .  Analyze  and  measure  any 
in  cost  or  progress 
development, 

6.  Identify  specific  factors 
accurate  cost  estimating. 

C.  RESEARCH  QUESTIONS 
Primary  Research  Question: 

How  do  estimated  development 

compare  with  actual  costs  for  the 
Subsidiary  Research  Questions: 

1.  What  methods  and  techniques  were  used  to 
estimate  development  and  production  costs? 

2.  How  was  technological  progress  measured  and 
evaluated? 

3.  What  factors  helped  or  hindered  accurate  cost 
estimating  and  accomplishment  of  stated  SOA 
extension  goals? 

D.  SCOPE  LIMITATIONS  AND  ASSUMPTIONS 

This  thesis  is  a  case  study  of  the  ANGAS  project. 
The  primary  task  is  to  obtain  and  analyze  estimated  and 
actual  cost  data,  document  cost  estimation  procedures 
for  the  ANGAS  project  and  to  evaluate  generic  cost 
estimating  procedures  used  by  Lockheed  for  SOA 
projects.  The  technological  requirements  of  the  ANGAS 
project  are  studied  to  measure  the  accomplishment  of 
stated  ANGAS  technological  goals. 

The  ANGAS  project,  an  ongoing  effort  at  Lockheed, 
is  designed  to  produce  one  end  unit  to  fly  aboard  a 
satellite.  Thus,  the  cost  data  and  estimation 
procedures  presented  in  this  paper  are  more  applicable 
to  development  costs  rather  than  to  production  costs. 
This  is  important  because  many  SOA  extensions,  by  their 


to  develop  the  cost 

o  estimated  costs, 

significant  variances 
in  technological 

that  help  or  hinder 


and  production  costs 
ANGAS  project? 
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nature,  must  be  implemented  through  a  substantial 

development  effort.  At  the  outset  of  many  SOA 

development  efforts,  the  end  unit  is  more  of  an 

abstract  idea  than  a  detailed  production  specification. 

In  the  1970’s,  system  acquisitions  programs  were 
usually  focused  on  specific  technical  approaches  at 
the  time  of  their  initiation. 

Today  a  program  is  initiated  after  competent 
authority,  the  Program  Decision  Authority,  approves  a 
specific.  formally  stated  mission  need,  based  on 
Mission  Area  Analysis,  submitted  within  the  first 
Program  Objectives  Memorandum  in  which  program  funds 
are  requested.  [REF.  2  :  p.  1-7. ] 

E.  METHODOLOGY 

The  methodology  used  in  this  research  is  case 
study.  Research  trips  to  Lockheed  included  visits  with 
the  ANGAS  contract  administrator,  principle 
investigator,  senior  cost  estimator  and  several 
development  engineers.  These  Lockheed  personnel  are 
located  at  facilities  in  Sunnyvale  and  Palo  Alto, 
California.  Lockheed  personnel  provided  estimated 
costs,  the  associated  cost  estimation  methods,  actual 
costs  incurred  to  date,  a  summary  of  progress, 
technological  breakthroughs  and  specific  factors  that 
have  helped  and  hindered  the  SOA  extension  effort. 

The  NAVPRO  office,  located  in  Sunnyvale,  was 
visited  numerous  times  to  identify  a  suitable  contract 
for  study  and  to  obtain  relevant  contract  and  cost 
data . 

The  ANGAS  program  manager  was  also  interviewed  to 
determine  what  the  government  hoped  to  achieve  from  the 
ANGAS  project  and  how  technological  progress  of  that 
project  is  measured. 

Background  and  research  data  were  obtained  through 
the  Naval  Postgraduate  School  library  and  its  research 
services.  Automated  searches  were  conducted  through 
the  NPS  library’s  DOD,  NON-DOD  and  Defense  Technical 
Information  Center’s  files.  Information  obtained 


through  visits  to  Lockheed  during  the  case  study  and 
related  research  material  is  contained  in  this  thesis. 


F.  SUMMARY  OF  FINDINGS 

Major  findings  of  this  thesis  are  summarized  below: 


1  . 


There  were  no  significant  variances  between 
estimated  an  actual  costs  to  date.  Spending 
was  at  a  slightly  lower  rate  than  anticipated. 


2  . 


Like  many  research  and  development  protects 
that  are  administered  by  a  "level  of  effort 
contract",  measurement  and  analysis  of 
technological  development  is  the  primary  gauge 
of  effectiveness  and  progress. 


3  . 


SOA  extensions,  achieved  by  numerous  smaller 
scale  projects  that  incrementally  extend  the 
SOA,  reduce  the  risk  of  both  costs  overruns  and 
failure  to  achieve  technological  goals. 


4  . 


Personnel  costs  are  the  most  significant  cost, 
factor  in  the  development  phase  of  SOA 
extensions. 


5  . 


project 


can 


Incremental  funding  of  a  DOD  SOA 
provides  leverage  to  the  government  that 
reduce  cost  overruns  and  achieve  desired 
technological  goals  but  tends  to  increase  the 
length  of  time  required  for  the  development 
effort . 


6  . 


Currently  available  parametric  models,  used  In 
private  Industry,  could  be  applied  to 
government  use  with  proper  historical  data 
application  and  case-by-case  analysis. 


G.  ORGANIZATION  OF  STUDY 

Chapter  II  provides  a  background  of  the  ANGAS 
project,  an  overview  of  cost  estimation  procedures  used 
for  ANGAS  and  a  summary  of  cost  estimation  procedures 
and  models  used  at  Lockheed. 

Chapter  III  details  the  methodology  used  in  this 
thesis  and  present  data. 

Chapter  IV  analyzes  the  data  in  chapter  III 
consistent  with  the  stated  objectives. 

Chapter  V  states  the  conclusions  drawn  from  this 
SOA  extension  case  study. 


ifl 

a 


» 


J-'  -■ 


V. 


II.  BACKGROUND 


A.  PROJECT  OVERVIEW 

The  Defense  Advanced  Research  Projects  Agency  -  201 
ANGAS  program  involves  the  design,  test  and 
fabrication  of  the  ANGAS  flight  instrument  and  the 
performance  of  a  long  term  flight  demonstration  in 
space.  [REF.  3  :  p.  3-2. J 

The  ANGAS  project  is  designed  to  develop  a  gamma- 
ray  spectrometer  that  will  be  carried  into  orbit  by 
satillite.  The  spectrometer  is  designed  to  pasively 
detect  radiation  in  space  througn  an  array  of  germanium 
sensors.  The  project  will  be  completed  when  the 
designed  spectrometer  has  been  flown  in  space  and  the 
specified  scientific  results  have  been  supplied  to  the 
government.  The  specific  requirements  of  the  ANGAS 
project  are  provided  in  the  Statements  of  Work  listed 
in  Appendix  B. 

The  ANGAS  system  required  development  of  a  large 
mul t 1 -element  array  of  new,  n-type,  segmented  germanium 
sensors.  The  design  concepts  represent  a  research  and 
development  effort  which  has  never  been  performed 
before.  This  SOA  extension  in  gamma-ray  spectrometers 
incorporates  the  following  new  technologies: 

1.  Imaging  collimator, 

2.  Cooled,  radiation  resistant  n-type  germanium 

sensors  , 

3.  Anticoincidence  shield, 

4.  Electronics  subsystem  which  enables  rapid 

onboard  data  analysis  and  microprocessor 
control , 

5.  Solid  cryogen  cooler. 

The  ANGAS  program  is  being  developed  by  Lockheed 
Space  Sciences  Laboratory  which  is  part  of  the  Lockheed 
Research  and  Development  Division,  the  principle 
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research  organization  of  the  Lockheed  Missiles  and 
Space  Corporation. 

The  government’s  program  manager  for  ANGAS  is  LTCO: 
George  P.  Lasche ,  USA.  He  is  in  the  Geophysical 
Sciences  division  at  the  Defense  Advanced  Research 
Projects  Office,  Arlington,  VA.  He  described  the  ANGAS 
program  goals  as: 

An  effort  to  develop  new  techniques  and  Improve 
sensitivity  and  capability  to  record  and  passively 
detect  radiation  in  space  through  .a  process  nf 
imaging  with  high  energy  resolution.1 

A  wide  variety  of  potential  DOD  uses  for  ANGAS 
include  its  use  as  a  means  to  verify  arms  treaty 
compliance  from  space. 

B.  OVERVIEW  OF  COST  ESTIMATION  PROCEDURES  USED  FOR 

ANGAS  CONTRACT  PROPOSALS 

The  principle  method  used  by  Lockheed  Space 
Sciences  Laboratory  to  determine  the  cost  estimates  for 
ANGAS  contract  proposals  is  a  bottom-up,  engineering 
estimate.  To  date  approximately  $4.96  million  has  been 
funded  in  two  Increments,  in  a  "level  of  effort"  type 
contract.  The  projected  total  funding  for  the  entire 
ANGAS  project  will  be  approximately  17  million  dollars. 

Dr.  George  Nakano,  Lockheed’s  principle 

investigator  (project  director  /  technical  supervisor) 

for  ANGAS  explained  that  the  ANGAS  cost  estimate: 

Is  a  bottom-up  estimate  that  is  developed  based  on 
past  experience  of  costs  ^incurred  developing  similar 
satellite  borne  systems. 

The  bottom-up  estimating  methodology  used  at 
Lockheed  provides  detailed  functional  and  cost  element 
estimates  provided  by  the  lowest  competent  level  in  the 
organization.  The  cost.  estimates  developed  for  each 

*  Information  was  obtained  by  telephone 

conversation  on  September  3,  1987. 

2  Information  was  obtained  in  an  interview  at  Palo 
Alto,  CA  on  September  4,  1987. 


phase  of  the  ANGAS  contract  were  based  on  historical 
cost  data,  the  requirements  of  the  Work  Breakdown 
Structure  (WBS)  and  the  Statement  of  Work  (SOW)  items 
listed  in  the  Requests  for  Proposal  (RFP). 

Lockheed’s  contract  pricing  proposal  for  ANGAS 
contained  the  following  summarized  items.  [REF.  3  :  p. 
2.2-2.3] 

1.  The  overhead,  general  and  administrative  rates 
are  applied  as  a  percent  times  direct  labor 
dollars.  The  rates  were  forecasts  based  on 
historical  data  and  current  projections. 

2.  Material  requirements  are  identified,  listed 
and  priced  by  component  or  subsystem.  These 
direct  purchase  items  range  from  firm  prices  to 
estimates  based  on  the  accuracy  that  program 
requirements  were  defined. 

3.  Procurement  burden  is  applied  as  a  percent  of 
the  total  estimated  costs  for  material, 
purchased  serices  and  subcontract  and  outside 
production  costs.  The  rates  are  based  on 
historical  data  and  current  projections. 

4.  Cost  of  money  is  an  element  of  facilities 
capital.  The  rates  are  applied  to  the  direct 
labor  dollars  associated  with  annual  overhead 
and  general  andadministrative  rates  and  to  the 
base  amount  associated  with  the  annual 
procurement  burden.  The  cost  of  money  rate  used 
was  10.375  percent. 

5.  Air  fare  rates  are  averages  of  recent  Lockheed 
ticket  costs  for  tourist  class  to  the 
designated  cities.  Per  diem  rates  are  based  on 
historical  data  and  current  projections. 

6.  Direct  labor  hours,  the  most  significant 
component  of  the  ANGAS  cost  estimate,  are  based 
on  a  technical  evaluation  developed  from  a 
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detailed  analysis  of  the  program  requirements. 
The  basic  labor  rates  are  current  average  rates 
for  direct  labor  pools,  by  skill  categories 
that  are  used  In  the  program.  These  rates  are 
escalated  by  approved  factors  when  appropriate 
to  allow  for  future  labor  rate  increases  during 
the  program. 

All  costs  and  applied  rates  were  audited  by  the 
Defense  Contract  Audit  Agency  and  negotiated  with  tin- 
cognizant  government  activity.  Specific  costs,  rates 
and  audit  results  will  be  discussed  In  Chapters  III  and 
I  V  . 

Labor  hour  and  cost  estimates  were  based  on 
previous  satellite  flight  programs  involving  similar 
but  much  smaller  gamma-ray  sensor  systems  developed  by 
Lockheed.  The  Defense  Advanced  Research  Projects  Agency 
301  program  although  only  one-forth  the  size  of  AN'GAS 
and  less  technically  complex  was  used  In  developing  the 
AN'GAS  cost  estimate. 

C.  SUMMARY  OF  COST  ESTIMATION  PROCEDURES  AND  MODELS 

USED  BY  LOCKHEED 

The  LMSC  Estimating  Systems  Description  (ESDI 
Manual  provides  company-wide  guid; nee  for  estimating 
requirements  for  government  or  prime  customer 
contracts . 

The  LMSC  ESD  Manual  states: 

overall  responsibility  for  the  LMSC  estimating 
definition,  development,  control  and  compliance  Is 
vested  in  the  Vice-President,  Finance.  The  Division 
Industrial  Accountants  report  to  the  Vice-President 
Finance  and  are  responsible  for  ensuring  compliance 
with  this  document.  [REF.  4:  pi. 3] 

The  following  summary  of  cost  estimation  procedures 
and  models  is  provided  from  Information  obtained  from  a 
presentation  on  SOA  pricing  by  Mr.  Ted  Castro,  Manager 
LMSC  Estimating  Systems,  Mr.  Donald  H.  Palmby,  Manager 
LMSC  Cost  Modeling  and  Analysis,  and  Mr.  Ken  Peeler. 


Chief  Industrial  Accountant  LMSC  Astronautics  Division 
and  the  LMSC  ESD  Manual . 


1 .  SOA  Proposal  Estimating  Process 

Figure  2-1,  which  follows  this  description  of 
Lockheed’s  contract  proposal  estimating  procedure 
provides  a  flow  diagram  of  the  process . [REF .  4  :  p. 

2.32  -  2.62] 

The  cost  kick-off  meeting  establishes  a 

schedule  of  deadlines  for  completion  of  cost  inputs. 

Cost  inputs  and  bases  are  generated  for  materials, 

direct  labor  and  other  direct  costs. 

The  Federal  Acquisition  Requirements  (FAR) 

directs  DOD  contractors  to: 

provide  a  consolidated  price  summary  of  individual 
material  quantities  included  in  various  tasks,  orders 
or  contract  line  items  being  proposed  and  the  basis 
for  pricing  (vendor  quotes,  invoice  prices,  etc.). 
[REF  5.  :  15.804-6] 

Material  requirements  are  broken  down  into 
three  categories: 

1.  Subcontract  estimates:  obtained  through  formal, 
detailed  RFPs  or  through  estimation  if 
sufficient  time  is  not  available  to  identify  a 
subcontractor  and  obtain  a  price  quote  before 
Lockheed  must  respond  to  the  government 
proposal.  Lockheed  is  required  to  establish  the 
reasonableness  of  any  estimate  through  an 
Independent  cost  analysis  or  technical  basis. 

2.  Material  estimates:  includes  prices  for 

standard  commercial  items,  raw  materials, 
purchased  parts  and  material  usage  costs.  These 
material  items  are  priced  through  historical 
data,  supplier  quotations  and  estimations. 

3.  Intra-Lockheed  Work  Transfer  ( IWT )  Items:  these 
material  items  are  made  at  Lockheed.  FAR 
requires  that  these  items  be  transferred  at 


cost  with  an  explanation  of  the  pricing  method 
used . 

Once  all  material  requirement  costs  are 
consolidated  they  are  reviewed  and  additional  analysis 
and  justification  is  provided  if  needed  or  required  by 
FAR.  A  consolidated  price  summary  of  all  material 
requirements  is  then  generated.  Lockheed’s  final  action 
for  material  costs  is  the  application  of  material 
escalation  rates  if  justified  for  a  particular  contract 
or  contract  phase. 

Direct  Labor  cost  estimates  must: 

provide  a  time-phased  (e.g.  monthly,  quar ter ly , etc . ) 

breakdown  of  labor  hours,  rates  and  cost  by 
ropriate,  category.  and  furnish  bases  for 
imates.  [REF.  5  :  FAR  15.804-6J 

Direct  labor  estimates  are  usually  expressed  by 

Lockheed  in  labor  hours,  identified  directly  to 

research,  design,  production  or  other  project 

objective.  To  justify  direct  labor  estimates,  the 

requirements  of  an  SOA  extension  RFP  are  broken  down 

into  tasks  consistent  with  SOW  requirements  or  WBS 

element.  The  two  basic  methods  used  to  provide  a  basis 

of  estimate  (BOE)  for  hours  quoted  are: 

1.  Historical  or  similar  to  data:  this  is  the 
preferred  BOE  for  quoting  direct  labor  hours. 
Similar  programs  are  used  to  identify  tasks 
that  are  equivalent  to  tasks  to  be  performed  in 
the  proposed  contract.  Adjustments  or  applied 
factors  are  often  used  to  calibrate  historical 
data.  Justification  of  adjustments  or  factors 
is  required  by  FAR. 

2.  Engineering  estimates:  used  when  no  historical 
or  similar  to  data  is  available.  Tasks  are 
broken  down  to  the  lowest  level  of  detail. 

Use  of  estimating  standards  is  a  direct  labor 
estimating  method  which  relates  production  costs  to 
specific  characteristics  of  a  product,  such  as  weight, 


size  or  composition. 

Cost  estimating  standards  must  be  applied 
consistently  based  on  verified.  uniformly  correlated 
data  expressed  in  the  same  unit  of  measure .[ REF .  4  : 

p .  2.60] 

A  learning  curve,  which  predicts  the  extent  to 
which  experience  in  performing  a  task  will  decrease  the 
labor  time  needed  to  perform  that  task,  is  then  applied 
by  Lockheed  to  all  labor  estimates.  The  LMSC  labor  cost 
accumulation  system  is  the  data  base  used  to  develop 
learning  curves.  Curve  selection  and  slope  definition 
depend  on  a  number  of  factors,  including: 

1.  how  the  data  has  been  accumulated  (by  lot  or 
unit), 

2.  whether  the  item  is  a  follow-on  to  an  existing 
production  run  or  an  estimate  based  on  a 
similar  item, 

3.  if  a  production  break,  change  in  design 
technology  or  factory  method  has  occurred,  and 

4.  customer  specification  of  the  methodology, 

[REF.  4  :  p  2.61] 

Once  total  labor  hour  estimates  are  compiled 

and  reviewed  Lockheed  direct  labor  hour  rates  are 

applied  to  the  labor  hours.  The  estimation  of  "other 

direct  costs"  and  BOE  is  the  final  major  cost  category 

that  must  be  derived  for  an  SOA  extension  price  quote. 

Other  direct  costs  include: 

travel,  overtime  premiums,  allocated  prime  cost 
(APC)/pooled  work  orders,  monitored  line  parts  APC , 
relocation  expenses  ana  foreign  field  service 

bonuses . [REF .  6  :  p.  2-62]. 

Other  direct  costs  are  usually  based  on 

historical  or  direct  estimating  procedures.  All 

estimates  for  materials,  direct  labor  and  other  direct 

costs  form  the  initial  cost  proposal  which  is  now 

reviewed  and  approved  or  disapproved.  If  approved 

management  price  review  and  negotiation  and  sometimes 

best  and  final  offer  steps  follow. 
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2 .  Bottom  -  Up  Cost  Estimating  Method 

Lockheed  uses  a  multiple  estimating  approach. 
When  detailed  task  definition  is  available  a  bottom  - 
up  /  engineering  estimate  is  prepared  by  the  cognizant 
division(s).  These  bottom  -  up  estimates  provide: 

1.  High  degree  of  accuracy  and  traceability. 


Cost  breakdowns  to  the  lowest  levels  of  the 
WBS , 


Based  on  historical  data. 


The  primary  disadvantages  of  a  bottom  -  up  cost 
estimate  for  an  SOA  extension  effort  are: 


Dependence  on  detailed  design  specifications 
which  do  not  usually  exist  during  early  phases 
of  SOA  extension  projects, 


Expensive  and  time  consuming. 


Cannot  be  used  as  the  sole  costing  method 
Trade  -  off  analysis  is  not  easily  evaluated. 


Although  many  SOA  extensions  do  not  initially 
provide  detailed  design  specifications,  bottom  -  up 
cost  estimating  is  sometimes  still  used  by  Lockheed. 
Estimates  are  based  on  the  experience  gained  by 
individual  engineers  and  technicians  from  developing 
systems  or  components  that  used  existing  SOA 
technology.  Many  of  the  SOA  extension  contracts  awarded 
to  Lockheed  are  an  extension  of  technologies  and 
systems  developed  previously  by  Lockheed.  Bottom  -  up 
SOA  extension  cost  estimates  are  reconciled  with 
experience  and  computer  cost  models. 

Various  computer  models  are  used  in  SOA  cost 
estimating  by  Lockheed  to  establish  "price  targets", 
provide  "sanity  checks"  and  reconcile  other  cost 


es  t imat ions 


Parametric  Cost  Models 


Lockheed  uses  the 


PRICE  Hardware 


parametric  cost  model  early  in  the  development  or 


concept  exploration  stages  of  an  SOA  extension. 

The  PRICE  model  contains  thousands  of  mathematical 
equations  relating  the  input  variables  to  cost.  Each 
specific  set  of  input  parameters  uniquely  defines  the 
hardware  for  cost  modeling.  The  resultant  cost  output 
is  determined  from  the  mathematical  equations  alone. 
PRICE  does  not  perform  the  function  of  a  table  look¬ 
up  model. 

PRICE  has  been  designed  to  estimate  costs  with  a 
minimal  amount  of  hardware  information.  This  feature 
makes  it  a  legitimate  tool  for  cost  estimation  of 
programs  in  concept  stage  of  development , since  the 
model  uses  its  internally  generated  values  for  any 
missing  input  variables  in  order  toestimate  cost.  Of 
course.it  is  always  preferable  for  the  PRICE  user  to 
supply  the  inputs,  when  their  values  are  known.  In 
this  way,  the  statistical  uncertainty  is  reduced. 
[REF.  6  :  p.  7] 

Lockheed  applies  the  PRICE  H  model  when  Skey 
representative  system  characteristics  are  known. 

PRICE  H  also  identifies  cost  drivers  and 
quickly  and  economically  evaluates  SOA  extension  trade¬ 
off  and  schedule  analysis. 

The  major  disadvantages  of  the  PRICE  H  model  as 
noted  by  LMSC’s  senior  cost  estimating  managers  are: 

1.  Not  based  on  actual  history, 

2.  Algorithms  need  to  be  adjusted  for  different 

projects,  divisions  and  acquisition 

env i ronments . 

PRICE  H  relies  on  input  of  a  subassembly  or 
higher  level  unit’s  active  electronic  weight  and 
mechanical / structural  weight  as  the  most  critical  cost 
and  schedule  factors.  The  accuracy  of  the  PRICE  output 
is  directly  related  to  the  accuracy  of  the  weights 
input . 

Input  characteristics  of  the  PRICE  H  model 
which  enhances  its  value  as  an  SOA  extension  cost 
estimation  model  are: 

1.  Inputs  consider  the  lead  time  necessary  for 

set-up,  parts  procurement  and  redesign.  This  is 
significant  for  most  SOA  extensions. 


3  Information  obtained  from  LMSC  SOA  cost 
presentation,  September  24,  1987,  Sunnyvale,  CA . 
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The  degree  of  new  design  and  complexity  of 
development  engineering  tasks  is  included. 

3.  The  operational  environment  in  which  the  system 
is  to  be  used  is  evaluated  in  cost  estimates. 
SOA  extensions  such  as  the  ANGAS  project  are 
designed  to  be  used  in  a  new  operational 
environment,  outside  the  earth’s  atmosphere. 

4.  Type  and  manufacturing  complexity  of  the 
electronics  within  the  system  or  component  is 
evaluated  by  the  model.  New  electronics  design 
is  often  a  significant  factor  of  SOA  extension 
costs . 

5.  The  model  also  evaluates  technological 
improvement,  year  of  technology  and 
technological  delay  or  lags.  These  Inputs  are 
often  SOA  extension  cost  drivers. 

Illustration  2-1  is  a  sample  output  of  a  PRICE 

H  run  provided  by  LMSC  Cost  Modeling  and  Analysis 

Manager.  The  output  has  been  broken  down  into  six 

sections.  The  following  general  output  descriptions  are 

summarized  from  the  PRICE  Executive  Handbook  [REF.  7  : 

p.  20].  Relevant  SOA  extension  output  data  is  also 

highlighted . 

Section  1:  identifies  time  of  the  run,  the  name  of 
the  item,  general  information  about  the 
item,  the  recurring  unit  production  cost 
and  monthly  production  rates.  Illus.  2-1 
identifies  the  item  as  a  pyro-kit,  to  be 
produced  in  a  quantity  of  one.  The  unit 
weight  was  input  at  7.96  pounds,  the  unit 
volume  was  .26  cubic  feet  and  mode  two 
designates  the  item  as  a  mechanical  item 
with  no  electronic  elements.  The  quantity 
/  next  higher  assembly  designator  is  a 
one  which  determines  the  number  of  units 
required  to  be  integrated  and  tested  at 
the  next  higher  assembly  level. 

Section  2:  lists  program  costs  in  engineering  and 

manufacturing  subtotals.  Illustration  2-1 
indicates  only  production  costs  of  1008 
dollars.  Development  costs  are  blank,  in¬ 
dicating  that  no  development  costs  are 
required.  Many  SOA  extensions  require  sig¬ 
nificant  development  costs.  The  ability  of 
PRICE  H  to  output  development  and 
production  costs  separately  is  useful  in 
analysis  of  technological  alternatives. 
Illus.  2-1  lists  engineering  costs  of 
$504.  These  costs  are  composed  of 
drafting  costs  of  $46.,  design  costs  of 
$250.,  project  management  costs  of  $148. 
and  data  costs  of  $60.  Manufacturing 
costs  total  $505 .  These  costs  are 
composed  of  production  costs  of  $453.  and 
tools  and  test  equipment  costs  of  $52. 

Total  engineering  and  manufacturing  costs 
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equal  $1008. 

provides  output  useful  for  analysis  of  the 
Inputs  with  known  standards.  Density, 
engineering  changes  and  mean  time  between 
failures  values  in  Illus.  2-1  are  followed 
by  an  asterisk.  This  indicates  that  these 
input  values  were  not  known  ana  calculated 
by  the  model.  Many  inputs  required  to 
estimate  the  costs  of  an  SOA  extension  are 
not  available  during  the  early  phases  of 
development.  The  ability  of  tne  PRICE  H 
model  to  derive  output  and  then  provide 
missing  input  is  ideal  for  SOA  extensions 
during  early  phases.  Illus.  2-1  shows 
mechanical  design  factors  that  were  input 
for  the  run.  The  manufacturing  complexity 
of  10.42  is  an  empiricaly  derived  value 
that  represents  the  product’s 
produc ib i 1 i ty  which  is  a  function  of 
material  type,  finished  density  and 
fabrication  methods.  The  mechanical 
integration  level  of  .7  represents  the 
level  of  mechanical  integration  and  test. 
This  factor  describes  the  level  of  effort 
required  for  the  integration  of  mechanical 
equipment  in  the  next  higher  assembly.  The 
platform  input  of  2.2  designates  the 
intended  operating  environment.  An  input 
of  2.0  or  more  represents  unmanned  or 
manned  space  operations.  The  year  of 
technology,  1985,  has  been  calculated  by 
the  program.  The  reliability  factor  of 
1.0  is  a  multiplier  used  to  deviate  the 
mean  time  between  failure  value  which  has 
been  calculated  by  the  model. 

outputs  development  and/or  production 
schedule  Information.  Illus.  2-1  Indicates 
that  the  dates  for  completion  of  the  first 
item  and  overall  completion  were  not  known 
before  the  run.  The  model  provides  valuable 
schedule  information  which  is  often 
difficult  to  determine  for  SOA  extensions. 

provides  additional,  in-depth  cost  output. 
The  model  again  provides  previously  unknown 
information  which  would  be  valuable 
for  SOA  extension  estimating.  First  unit 
production  costs  are  $452.17,  amortized 
unit  costs  are  $1008.30.  the  production 
tooling  cost  factor  is  1.0  and  the  unit 
learning  curve  is  86.4  percent. 

The  production  cost  multiplier  of  1.20  is  a 
multiplying  factor  used  to  Include  mock-ups 
for  general  and  administrative  and  fee  or 
profit  in  the  production  cost  outputs. 

one  of  the  most  useful  outputs  of  the 
PRICE  H  model  is  the  cost  uncertainty 
measure.  Illus.  2-1  provides  a  cost  range 
of  954  to  1046  dollars.  SOA  extension 
development  and  production  costs  inherently 
contain  a  high  level  of  uncertainty 
due  to  the  technical  innovation  and 
engineering  complexity. 


The  Lockheed  STAR  Cost  Estimating  Model 
Lockheed’s  STAR  cost  estimation  model  combines 
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the  characteristics  of  a  similar-to  model  and  an 
associated  model,  A  similar-to  methodology  uses 
historical  cost  data  from  tasks  or  equipment  similar  to 
the  equipment  under  evaluation.  A  high  degree  of 
accuracy  and  cost  breakdowns  to  the  subsystem  or  box 
level  (WBS  level  3  or  4  )  is  provided. 

The  associated  program  methodology  uses  total 
annual  costs  from  technically  representative  projects 
that  have  been  adjusted  by  economic  and  complexity 
f  actors . 

The  basis  of  the  STAR  model  is  the  detailed 
historical  database  that  has  been  developed  over  a  20 
year  period.  This  data  base  represents  actual 
production  and  development  cost  experience,  rather  than 
success  oriented,  or  tailored  data. 

The  STAR  model  is  made  up  of  three  distinct 
estimating  models  with  separate  data  bases  for  each.  A 
flight  hardware  cost  estimator,  a  ground  hardware  cost 
estimator  and  a  software  cost  estimator.  The  individual 
models  use  an  algorithm  designed  for  its  particular 
line  of  business.  The  software  cost  estimator  works 
with  a  software  size  estimator  that  calculates  lines  of 
code  by  functional  application.  The  three  models  can  be 
used  in  an  integrated  manner  to  derive  a  total  system 
cost . 

Minimum  inputs  Include:  program  hardware  / 
software  lists,  a  master  schedule  and  the  year  dollars 
required.  Critical  SOA  extension  estimating  factors 
such  as:  design  complexity,  escalation  of  costs,  and 
quantity  are  incorporated  in  the  STAR  model. 

The  primary  advantage  of  the  STAR  model 
compared  to  the  PRICE  H  model  is  that  output  is  derived 
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from  actual  or  historical  costs.  STAR  also  allows 
updating  of  costs  as  a  project  progresses  and  tradeoff 
and  "what  If  analysis". 

The  STAR  model  contains  a  program  loader  that 
pulls  the  required  type  of  data  from  the  data  base.  The 
FILGEN  pulls  analogous  cost  items  from  the  data  base. 
Each  item  on  the  program  hardware  list  is  coded  with  a 
box  status,  box  type  and  engineering  status. 

Box  status  codes  are  either:  make  (m),  buy  (b) 
or  government  furnished  equipment  (g). 

Box  type  codes  are  either:  development  (d), 
qualification  (q)  or  production  (p). 

Engineering  status  codes  are  either:  new  (n), 
modified  (m),  existing  (e)  or  production  (p). 

Each  item  on  the  program  hardware  list  is  also 
assigned  input  values  for:  quality,  management  effort, 
design  engineering  required,  systems  engineering 
required,  software  engineering  effort  required, the 
manufacturing  complexity,  test  requirements  and  product 
assurance . 

The  model  now  makes  a  cost  run,  the  Lockheed 
labor  rates  are  the  default  labor  rate  values.  Appendix 
C4  provides  sample  output  for  a  subsystem  program 
hardware  list. 

Final  output  includes:  total  hours,  dollar 
value  for  the  labor  hours,  material  costs  in  dollars 
and  total  dollar  value  for  each  item  on  the  subsystem 
hardware  list.  The  final  total  price  is  obtained  from 
analogous,  historical  cost  data.  If  any  of  seven  input 
values  were  not  initially  known  the  total  cost  is 
normalized  and  reallocated  back  to  the  missing  aoxes. 

4  APPENDIX  C  is  not  part  of  the  unlimited 
distribution  of  this  thesis.  Individuals  within  DOD  may 
obtain  a  copy  of  APPENDIX  C  from  Professor  Willis  R. 
Greer,  Jr.,  Naval  Postgraduate  School,  Monterey, 
California. 


The  STAR  model  is  an  ideal  tool  for  Lockheed  to 
use  to  estimate  SOA  extension  costs.  The  data  base  is 
retrievable  by  line  item  or  box  level,  which  enables 
composite  data  from  even  classified  projects  to  be 
included  as  unclassified  useable  data. 

If  a  particular  line  item  on  the  hardware  parts 
requirement  list  is  going  to  be  subcontracted  or 
purchased  directly,  the  program  allows  these  known 
costs  to  be  entered.  Otherwise  a  data  base  is  selected 
which  matches  the  functional  and  characteristic 
qualities  of  the  item.  The  default  learning  curve 
value  is  95  percent. 

The  output  of  the  STAR  model  is  designed  to  aid 
SOA  cost  analysis.  Output  can  be  formatted  to  provide: 
development/non-recurring  costs  and  production  / 
recurring  costs.  Appendix  C  provides  sample  output  in 
this  format  and  an  example  of  output  in  WBS  reference 
number  sequence. 

The  accurate,  uniform  measurement  and 
correlation  of  historical  data  Into  box  level  or 
component  factor  enables  extremely  precise  SOA 
extension  cost  estimations  by  the  STAR  model  once  parts 
requirements  are  well  defined. 


Figure  2-1  Cost  Estimation 

Process 
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If  estimate  13  approved,  enter 
management  review  phase 


ILLUSTRATION  2-1 


I  . 

INPUT  FILENAME:  EX  24  SEPT  87  16:56 
GLOBAL  FILENAME:  LMSCGL01 

PYRO  KIT 

PRODUCTION  QUANTITY  1 
UNIT  WEIGHT  7.96  MODE  2 

UNIT  VOLUME  0.26  QUANTITY/NHA  1 

UNIT  PRODUCTION  COST  452.67 

MONTHLY  PRODUCTION  RATE  0.00 


TOTAL  COST 

46 

250 

148 

60 

504 

453 

52 

505 

1008 


PROGRAM 
( S1000  ) 
ENG 


COST 


DEVELOPMENT  PRODUCTION 


INEERING 
Drafting 
Design 
Systems 
Project  Mgmt 
Data 

Subtotal (ENG) 


MANUFACTURING 
Production 
Prototype 
Tool-Test  Eq 
Subtotal (MFG ) 


TOTAL  COST 


1008 


III  . 

DESIGN  FACTORS  MECHANICAL  PRODUCT  DESCRIPTORS 

Weight  7.960  Platform  2.20 

Density  30.038*  Year  of  Tech.  198 

Mfg.  Complexity  10.422  Reliability  Fac .  1. 

Engineering  Change  0.033*  MTBF(Field)  21971 

Integration  Level  0.700 


IV. 

SCHEDULE  START  FIRST  ITEM  FINISH 

Production  Jan  85  (19)  Jul  86*  (0)  Jul  86*  (19) 


OL.OO* 


ILLUSTRATION'  2-1  (Continued) 


V. 


SUPPLEMENTAL  INFORMATION 


Economic  Base 
Escalat ion 
T-l  Cost 
Amor  Unit  Cost 
Prod  Cost  Mult 


185  Tooling  and  Process 
0.00  Production  Tooling 
452.17"Rate  Tooling 
1 0 08 . 30 * P r i ce  Info  Factor 
1.20  Unit  Learning  Curve 


F  a  c  t  o  r  s 
1.00" 

0 

0.050 
0  .  S6  4 


VI  . 

COST  RANGES 
F  rom 
Center 
To 


DEVELOPMENT 


PRODUCTION 

954 

1008 

1046 


TOTAL  COST 
954 
1  008 
1046 


III.  METHODOLOGY  AND  DATA 


A.  INTRODUCTION 

This  chapter  describes  the  methodology  of  the 
thesis  and  presents  data  that  was  collected  during  the 
study.  The  methodology  section  is  divided  into  two 
subsections,  data  collection  and  data  analysis. 

Data  includes:  estimated  costs  submitted  by 
Lockheed  in  the  ANGAS  contract  proposals;  Defense 
Contracts  Audit  Agency  (DCAA)  audits  of  the  ANGAS 
contract  proposals;  costs  questioned  in  the  DCAA 
audits:  final  negotiated  costs  with  justifications; 
actual  costs  incurred  at  different  stages  of  the 
contract:  statement  of  work  items;  a  listing  of  major 
mi lestones . 

B.  METHODOLOGY 

1 .  Data  Collection 

The  methodology  used  in  this  research  is  case 
study.  An  information  search  was  conducted  to  obtain 
background  data.  Automated  searches  were  conducted 
through  the  Naval  Postgraduate  School  library’s  DOD, 
NON-DOD  and  Defense  Technical  Information  Center’s 
files.  The  following  literature  provides  information 
on  SOA  extensions,  development  and  production  cost 
estimation  Issues  and  a  background  that  will  enhance 
understanding  of  the  issues  in  this  case  study. 
Literature  that  is  pertinent  to  data  analysis 
methodology  will  be  presented  later  in  this  chapter. 

Captain  Helmut  W.F.  Scheel,  USAF ,  in  his 
master’s  thesis  [REF.  l:p.  1-91],  studied  the  methods, 
techniques  and  objectives  used  in  estimating  costs  for 
exploratory  development  projects  by  Air  Force  Research 
and  Development  Laboratories.  He  also  attempted  to 
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Identify  factors  which  contribute  to  the  variance 
between  project  manager’s  cost  estimates  and 
contractor’s  proposal  costs.  Captain  H.  Scheel 

surveyed  45  government  project  managers  to  determine 
cost  estimating  techniques.  Important  conclusions  of 
his  study  include: 

1.  77.8  percent  of  the  program  managers  used 

historical  data  for  general  cost  estimating. 

2.  82.2  percent  of  the  program  managers  used 

historical  data  and  experience/ Judgment  to 
estimate  labor  costs  and  manpower  requirements. 

3.  Low  cost  estimates  were  sometimes  made  because 
of  limited  funds  available  for  a  specific 
proj  ect . 

4.  The  major  variance  factor  attributable  to  the 
bidders  was  errors  in  estimation  as  a  result  of 
misinterpretation  of  the  Statement  of  Work 
(SOW). 

5.  Contractors  can  sometimes  propose  a  different 

level  of  effort  than  specified  in  an  SOA 
extension  due  to  technical  insight  and 

capabilities  known  only  by  the  contractor. 

6.  Establishment  of  a  computerized  system  with 
standardized  software  would  provide  a  data  bank 
that  would  enable  more  accurate  cost  estimation 
analysis  within  the  government. 


Most  project  managers  do  not  accurately 
estimate  costs  for  new  projects.  The  maioritv 
underestimate  by  30  percent  or  less,  although 
some  underestimate  by  as  much  as  200  percent. 
[REF.  1 : p .  33,  36,  41,  51,  67,  92-96] 


The  ANGAS  project,  like  most  SOA  extension 
efforts,  requires  a  long  research  and  development  (R&D) 
period.  To  accurately  estimate  and  evaluate  proposed 
contractor  costs  an  understanding  of  long-range 
corporate  R&D  planning  is  useful.  Ward  C.  Lowe  framed 


the  process  into  seven  significant  steps: 

1.  Specify  as  clearly  as  possible  the  basic 
technological  objectives  which  are  of  primary 
interest.  These  may  be  stated  broadly  at  the 
beginning  but  must  eventually  be  broken  down 
into  particular  areas. 


2.  Identify  the  goals  toward  which  the  company  is 
working,  which  ideally  are  set  forth  in  the 
corporate  and  the  research  objectives. 


3.  In  keeping  with  the  preceding  two  factors. 

seriously  and  imaginatively  consider  all 
possible  results  which  may  be  achieved  if  the 
research  efforts  are  successful. 


4.  Rank  the  hypothetical  capabilities  of  the 
research  efforts  as  determined  in  the  preceding 
step,  in  terms  of  their  potential  contribution 
to  the  achievement  of  specific  goals.  The 
ranking  should  achieve  the  corporate  or  the 
research  objectives. 

5.  Outline  the  principle  technological  steps  which 
are  required  to  achieve  the  hypothetical 
research  results  listed  in  the  preceding  step. 
Such  an  outline  should  uncover  the  more 
significant  gaps  in  existing  knowledge. 

6.  Select  the  small  number  of  high-value  research 
results,  as  determined  previously,  which  have 
reasonably  well-specified  steps  and  a  minimum 
number  of  knowledge  gaps.  Give  these  results 
further  examination  and  consideration  for  full- 
scale  research  efforts. 

7.  Remain  alert  to  any  developments  that  could 

significantly  change  the  value  ratings  used  in 
the  preceding  steps.  When  research 

breakthroughs  fill  in  missing  knowledge  gaps, 
the  process  should  be  repeated.  [REF.  8:p.  62- 
6  3  J 

The  above  steps  are  also  useful  for  government 
program  managers  when  defining  the  technological  goals 
of  SOA  extension  projects  and  drafting  the  SOW. 
Accomplishment  of  SOW  items  during  the  development 
phase  of  an  SOA  extension  contract  may  be  the  primary 
measurement  tool  for  evaluating  technological  progress. 

The  bottom-up,  parametric  and  simi lar-to-cost 
estimating  models  described  in  Chapter  II  are  the  most 
widely  used  cost  estimating  methods.  References  4,  6, 
and  7  provide  information  for  further  study. 

It  was  time  consuming  and  difficult  to  identify 
a  company  willing  to  cooperate  in  a  case  study  which 
offered  an  SOA  extension  project  useful  for  analysis. 
Several  Marketing  and  Research  and  Development 
Directors  were  contacted  by  telephone  to  determine 
interest  in  and  suitability  for  study. 

Companies  initially  contacted  included  GTE 
Inc.;  National  Semi-Conductor  Inc.;  Fairchild  Inc.; 
Advanced  Microcircuit  Devices  Inc.;  Cypress  Electronics 
Inc.;  Acrian  Inc.;  Hewlett  Packard  Inc.;  Harris 
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Microwave  Inc;  Intel  Corp.  Inc.;  Litton  Industries 
Inc.;  and  Eaton  Semiconductors  Inc. 

Company  responses  ranged  from  "not  interested" 
to  enthusiastic  requests  for  additional  information 
about  the  research  effort.  Companies  expressing  an 
interest  in  aiding  the  research  effort  were  then 
evaluated  on  their  ability  to  provide  viable  projects 
for  study.  Three  other  Naval  Postgraduate  School 
students  had  selected  SOA  extension  case  studies  for 
their  theses.  All  of  us  worked  as  a  team  to  identify 
appropriate  firms.  These  efforts  resulted  in  the 
selection  of  GTE  Inc.  and  Litton  Industries,  Inc. ,  as 
candidates  for  study  by  two  other  graduate  students. 

LCDR .  Fred  Voellm,  SC,  USN ,  of  the  Navy 
Procurement  Office  located  at  LMSC ,  was  contacted  for 
assistance  in  identifying  a  government  contract 
administered  by  his  office  for  an  SOA  extension  that 
could  be  studied.  LCDR.  Voellm  produced  a  list  of 
contracts  that  were  considered  high  technology  and 
potential  candidates  for  study. 

Individual  contracts  were  evaluated  on  the 
degree  of  new  technologies  and  SOA  extension,  the 
availability  of  initial  and  actual  cost  data  and  the 
stage  of  completion  of  the  contract. 

The  ANGAS  experiment  program  initiated  by  the 
Defense  Advanced  Research  Projects  Office  in 
conjunction  with  the  Research  and  Development  Division 
of  LMSC  was  selected  because  it  combined  numerous  SOA 
technological  extensions  into  a  design  that  was  unique 
and  required  operation  in  space.  Selection  of  the 
ANGAS  project,  which  is  still  in  early  research  and 
design  stages,  provided  an  opportunity  to  study 
development  costs  and  how  technological  progress  is 
evaluated . 
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Mr.  Don  Dorsett,  LMSC  ’  s  ANGAS  contract, 
administrator,  was  visited  numerous  times  to  obtain 
background  and  cost  data.  The  ANGAS  "Weekly  Financial 
Status  Report"  and  "Work  in  Process  Ledger  by  Work 
Order"  provided  actual  current  costs  and  spending 
rates.  A  summary  of  this  data  will  be  presented  in  the 
data  section  of  this  chapter.  Mr.  Dorsett  also 
discussed  the  significant  technological  and  SOA 
extension  characteristics  of  the  ANGAS  project.  These 
are  listed  in  Chapter  II. 

LTCOL.  George  P.  Lasche,  USA,  Geophysical 
Sciences  Division,  Defense  Advanced  Research  Projects 
Office,  Arlington,  VA ,  is  the  government’s  program 
manager  for  the  ANGAS  program.  LTCOL  Lasche,  USA,  was 
interviewed  by  telephone.  The  project’s  technological 
objectives,  goals,  potential  applications,  program 
status  and  evaluation  techniques  used  to  analyze 
proposed  contractor  costs  will  be  presented  in  the  data 
section  of  this  chapter. 

The  government  program  manager  for  the  ANGAS 
program  authorized  access  to  the  principle  investigator 
at  the  Lockheed  Space  Sciences  Laboratory  in  Palo  Alto, 
CA,  Dr.  George  Nakano.  Lockheed’s  principle 
investigator  for  the  ANGAS  project  is  responsible  for 
technical  supervision  and  overall  project  direction. 
Dr.  Nakano  was  interviewed  by  telephone  and  visited  at 
the  Space  Sciences  Laboratory.  His  description  of  the 
ANGAS  cost  estimation  effort,  accomplishment  of 
technological  goals,  spending  status,  problem  areas  and 
significant  SOA  extension  achievements  will  be 
presented  in  the  data  section  of  this  chapter,  Appendix 
B  and  Chapter  IV. 

Cost  estimation  procedures  used  by  Lockheed 
were  obtained  from  Lockheed’s  Estimating  Systems 
Description  Manual  and  from  a  presentation  on  SOA 


pricing  by  Mr.  Ted  Castro,  Manager  of  Lockheed’s 
Estimating  Systems;  Mr.  Donald  H.  Palmby,  Manager  of 
Lockheed’s  Cost  Modeling  and  Analysis;  and  Mr.  Ken 
Peeler,  Chief  Industrial  Accountant  of  Lockheed’s 
Astronautics  Division. 

Information  and  characteristics  of  bottom-up 
cost  estimating,  parametric  cost  models  and  Lockheed’s 
internally  developed  STAR  cost  estimation  were 
obtained.  Input  parameters  and  output  data  were 
obtained  for  sample  SOA  extension  component  cost 
estimates  using  the  parametric  and  STAR  cost  estimation 
models.  This  information  was  detailed  in  Chapter  II. 

2 .  Data  Analysis 

The  estimated  ANGAS  development  costs  as 
provided  in  Lockheed’s  contract  proposal  and  modified 
after  government  audit  and  negotiation  are  compared  to 
actual  reported  costs.  Differences  and  trends  in 
specific  cost  categories  are  analyzed.  Overall 
spending  rates  are  compared  with  anticipated  spending. 
The  results  of  the  data  analysis  are  presented  in 
Chapter  IV. 

Significant  characteristics  of  Lockheed’s  cost 
estimating  methods  and  models  that  are  potentially 
useful  in  aiding  better  estimation  of  SOA  extension 
projects  by  the  government  are  also  recorded  in  Chapter 
IV. 

The  specific  methodology  that  will  be  used  in 
the  cost  analysis  will  closely  follow  Dearden ’ s  simple 
variance  analysis  as  detailed  in  Cost  and  Budget 
Analysis.  Actual  costs  will  be  obtained  from  the  AXGAS 
"Weekly  Financial  Status  Report"  and  compared  with  the 
estimated  or  "budgeted  costs"  in  a  variance  matrix 
format . 

Budgeted  costs  will  define  the  planned 
expenditure  for  work  scheduled  for  accomplishment 


during  a  specific  phase  of  the  contract.  The  budgeted 
cost  from  work  scheduled  (BOWS)  for  a  level  of  effort 
contract,  like  the  ANGAS  contract,  "is  based  on 
expected  expenditures  by  tasks  to  be  completed."  [REF. 

9  :  p  .  6  ] 

A  determination  of  the  degree  of  completion  for 
stated  technological  goals  and  task  completion  is 
provided  in  Chapter  IV.  Information  obtained  from  the 
government’s  program  manager  and  the  ANGAS  principle 
investigator  about  the  status  of  the  SOW  items  will  be 
the  basis  for  establishing  task  and  technological 
progress.  The  reasons  why  the  program  manager  was  able 
to  : 

specify  as  clearly  as  possible  the  basic 
technological  objectives  which  are  of  primary 
Interest  [REF.  8:p.  62] 

will  also  be  discussed  in  Chapter  IV. 

C .  DATA 

1 .  Introduction 

The  Research  and  Development  Division  of  LMSC , 

Palo  Alto,  CA,  has  been  funded  through  an  incremental 

cost-plus-fixed-fee  contract  to  develop  the  ANGAS.  The 

major  milestones  are  presented  in  Appendix  B. 

In  a  cost-plus-fixed-fee  contract,  the  government 
pays  allowable  cost  and  a  fixed  fee.  Fixed  fee  does 
not  vary  with  actual  costs.  Provides  minimum 

incentive  for  contractors  to  control  costs. 
Completion  requires  contractor  to  deliver  end 
product.  Requires  specified  level  of  effort  over 
stated  period  of  time.  Used  most  widely  for 
research,  preliminary  exploration  or  study.  [REF. 
1 0 : p .  49] 

The  following  information  details  ANGAS  funding  to 
date  : 

PERIOD _ EST.  COST _ FIXED  FEE  COST  PLUS  FEE 

1  $  1,281,655  $  68,345  S  1,350,000 

2  S  3,392,620  S  192,843  S  3,585,463 

TOTALS  S  4,674,275  S  261,188  S  4,935,463 
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LMSC  submitted  a  contract  proposal  on  November 
26,  1985,  after  the  first  incremental  funding  had  been 
approved,  that  covered  a  78  month  period  commencing 
April  1,  1986  through  completion.  The  government 
choose  not  to  allocate  funds  for  this  entire  6  1/2  year 
period  and  has  incrementally  funded  the  project.  The 
contractor’s  proposed  cost  for  completion  of  ANGAS  was 
S16 , 589 , 398 . 00  (CPFF).  Appendix  B  contains  the  SOW 
items  from  the  two  incrementally  funded  contracts  and 
the  unfunded  proposed  completion  contract. 

D.  ESTIMATED  COSTS  FOR  FIRST  CONTRACT  PHASE 

LMSC,  Research  and  Development  Division  submitted 
the  following  breakdown  of  proposed  costs  for  the 
initial  ANGAS  contract: 


PRICE  BREAKDOWN 


DESCRIPTION 
LABOR  HOURS 
LABOR  DOLLARS 
OVERHEAD 
G  &  A  EXPENSES 
MATERIALS 
SUBCONTRACT 
PROCUREMENT  BURDEN 
TRAVEL 

SUB.  COSTS 
COST  OF  MONEY 
TOTAL  COST 
FIXED 

TOTAL  CPFF 


TOTALS 
14 , 688 
S  319,630 
506 , 192 
134 , 942 
110,793 
146,700 
9,012 
16 , 064 
SI , 243 , 333 
38 , 322 
SI , 281 ,655 
99,647 
Si  . 381 , 122 


The  individual  cost  element  totals  were 


developed  as  follows: 


Direct  Labor  -  direct  labor  hours  are  based  on 
a  technical  evaluation  developed  from  detailed 
analysis  of  the  program  requirements.  The 
labor  rates  are  current  averages  for  direct 
labor  pools,  by  skill  categories. 


> 


HOURS 


RATE 


AMOUNT 


DESIGN 

DESIGN 

DESIGN 


ENGINEER 
ENGINEER  - 
SPECIALIST 


SENIOR 


ENGINEER? SC  I ENT IF I C  CLERICAL  AID 
LABORATORY  ANALYST  -  RESEARCH 


RESEARCH  ENGINEER 
SCIENTIST,  SENIOR 
SCIENTIST  -  RESEARCH 
STAFF  ENGINEER 
STAFF  SCIENTIST 
TECHNICAL  PUBLICATIONS 


TOTAL  DIRECT  LABOR 


688  $21 . 76  $319,630 


Overhead  rates  are  applied  as  a  percent  times 
direct  labor  dollars.  The  rates  used  for  1985 
and  1986  were  1.592  and  1.560,  respectively. 
These  rates  apply  to  R  &  D  projects. 


BASE 


RATE 


AMOUNT 


OVERHEAD 


1985 

1986 


$236 ,523 
$83 ,107 


1 . 592 
1 . 560 


TOTAL  OVERHEAD 


$319 , 630 


$506 ,192 


General 
applied 
dollars . 
.424  and 


and  Administrative  Expenses  are  also 
as  a  percent  times  direct  labor 
The  rate  used  for  1985  and  1986  were 
.417,  respectively. 


BASE 


RATE 


AMOUNT 


G  &  A  EXPENSES 


1985 

1986 


>236,523 
i  83,107 


0 . 424 
0 .417 


;100 , 286 
;  34 ,656 


TOTAL  G  &  A  EXPENSES 


$319,630 


$134,942 


Material  requirements  were  listed,  identified 
and  priced  by  component.  Costs  of  purchased 
services  are  also  included  in  tnis  cost 
element . 


MATERIAL 
PURCHASED  SERV. 


1985 

1985 


151 ,547 
>59 , 246 


TOTAL  MATERIALS 


$110,793 


Subcontract  cost  estimates  are  based  on  actual 
price  quotes  received  from  the  subcontractors 
in  response  to  a  RFP .  Subcontract  costs  for 
1985  were  estimated  at  $146,700. 


Procurement  burden  is  applied  at  the  rate  of 
.035  of  the  total  estimated  costs  for 
materials,  purchased  services  and  subcontract 
costs.  The  rate  is  developed  from  historical 
data . 


BASE 


RATE  AMOUNT 


1985 


257,493 


TOTAL  PROCUREMENT  BURDEN 


0.035  $9,012 

$9,012 


Travel  -  per  diem  rates  are  based  on  historical 
data  and  air  fare  costs  are  based  on  tourist 


class  rates.  Travel  reflects  anticipated 

project  requirements. 


AIR  FARE 

ALBUQUERQUE 

2  TRIPS  OF  2  MEN  FOR 
TECH  MEETING 

3  DAYS 

1985 

QTY 

4 

RATE 

$  193 

AMOUNT 

772 

DENVER 

2  TRIPS  OF  2  MEN  FOR  3  DAYS 
INTEGRATION  COORDINATION 

1985 

4 

$  325 

1 , 300 

LOS  ANGELES/ BURBANK 

3  TRIPS  OF  2  MEN  FOR 
FLT .  COORD.  MEETING 

1  DAY 

1985 

4 

$  145 

870 

WASHINGTON/ BALTIMORE 
6  TRIPS  OF  2  MEN  FOR 
VENDOR  &  CUST  COORD. 

3  DAYS 

1985 

12 

$  556 

6 , 672 

PER  DIEM 

ONE  DAY  TRIPS 

WEST  COAST  TRIPS 

EAST  COAST  TRIPS 

6 

24 

36 

$  45 
$  91 
$  111 

270 

2 , 184 

3 , 966 

TOTAL  TRAVEL 

$ 

16 , 064 

8.  Cost  of  money  is 

capi tal .  Rates 

dollars,  general 
procurement  burden 

an 

are 

and 

base 

element 
applied  t 
overhead 

of  facilities 
o  direct  labor 
base  and  the 

DEVELOPMENT- LMSC  1985  $ 

1986  | 

G  &  A  EXPENSES  1985  $ 

1986  S 

PROCUREMENT  BURD  1985  $ 

TOTAL  COST  OF  MONEY 


BASE 


236,523 
83 ,107 
236,523 
83,107 
257 ,493 


RATE 

0 .10310 
0 .12283 
0.01055 
0 . 01254 
0 . 00074 


AMOUNT 

124 , 386 
10 ,208 
2 ,495 
1 , 042 
_ 191 

S  38.322 


The  total  of  the  cost  elements  is  $1,281,655. 
Lockheed  submitted  the  contract  proposal  with  a  fixed 
fee  of  $99,467,  8  percent  of  costs.  The  total  CPFF 
requested  by  Lockheed  for  the  first  phase  of  the 
contract  was  $1,381,122. 


E.  DCAA  AUDIT  REVIEW 


Lockheed’s  cost  proposal  was  audited  by  DCAA, 
Sunnyvale,  CA.  The  results  summarized  below  are  set 


COST 

ELEMENT 


CONTRACTORS 

PROPOSAL 


QUESTIONED 

COSTS 


SUBCONTRACTS 
PURCHASED  SERVICES 
MATERIAL 

PROCUREMENT  BURDEN 


S  146,700 
59,246 
51,547 
9,012 


S 


25 , 379 
59,246 
51 ,547 
4  ,766 


S  266,505 
subcontract 


$  140,938 
costs  of  S 


DCAA  questioned  the 
146,700  because  a  cost  or  price  analysis  had  not  been 
completed  by  LMSC.  Cost  or  price  analysis  for 
subcontracts  is  required  by  FAR  15.806.  DCAA, 

therefore,  applied  a  17.3  percent  negotiation  reduction 
factor  to  the  proposed  costs.  The  17.3  percent  factor 
is  based  on  DCAA  evaluation  of  subcontract  history  in 
the  Astronautics  and  Space  Systems  Division. 

The  purchased  services  of  $  59,246  and  material 
of  $51,547  were  questioned  because  LMSC  did  not  provide 
any  data  in  support  of  the  proposed  costs. 

The  procurement  burden  was  reduced  due  to  the 
above  questioned  costs  and  the  reduced  procurement 
burden  base. 

QUESTIONED  BASE  COSTS  $136,172 
PROPOSED  RATE  X  3.5% 

TOTAL  QUESTIONED  $  4,766 


1 .  Summary  Of  Cost  And  Fee  Negotiations 

LMSC  submitted  an  SF  1411  with  supporting 
documentation  to  justify  the  questioned  subcontractor 
costs.  This  documentation  was  reviewed  and  approved  by 
the  negotiator  and  scientific  officer. 

LMSC  also  provided  a  detailed  breakdown  of  the 
questioned  costs  for  purchased  services  and  materials 
which  was  approved  by  the  government.  These  prj.ce 
breakdowns  were  based  on  past  experience  on  similar 
projects,  such  as  project  Winkler,  catalog  prices  and 
vendor  quotes. 

All  questioned  costs  were  reviewed  and  accepted 
after  negotiation.  LMSC  had  requested  a  fee  of  S 


percent  of  costs  ($99,467),  the  government’s  fee 
objective  was  4.2  percent.  As  a  result  of 

negotiations,  the  settled  fee  was  5.5  percent 
($68,345).  The  final  estimated  costs  for  the  first 
phase  of  the  ANGAS  project  was: 

ESTIMATED  COST  FIXED  FEE  TCFF 

$1,281,000  $  68,345  $1,350,000 


F.  ESTIMATED  COSTS  FOR  SECOND  CONTRACT  PHASE 

LMSC  submitted  a  cost  proposal  dated  26  November 
1985  in  the  amount  of  $16,589,398,  covering  a  period  of 
performance  of  six  and  one-half  years.  Audit  report 
7481  -  6G210136  conducted  by  DCAA ,  Sunnyvale,  CA , 

questioned  portions  of  every  item  in  the  budget.  LMSC 
was  unable  to  substantiate  its  cost  estimates  to  the 
satisfaction  of  the  government.  LMSC  was  unable  to 
accurately  project  costs  six  years  into  the  future  for 
a  program  still  in  the  design  phase. 

It  was  decided  to  contract  only  for  the  final 
design  and  sensor  shield  evaluation  of  the  ANGAS 
project.  The  SOW  is  illustrated  in  Appendix  B.  LMSC 
proposed  a  cost  of  $  3,694,580.  The  costs  were  broken 
down  into  the  same  cost  elements  as  the  first  contract 
detailed  earlier  in  this  chapter. 

1 .  DCAA  Audit  Review  And  Negotiation 

The  following  proposed  costs  were  questioned 
and  negotiated: 

1.  Labor  Rates  -  LMSC  used  an  escalation  factor  of 
4.5  percent.  Due  to  the  government  ceiling  on 
salary  escalation,  LMSC  was  instructed  to 
recompute  labor  rates  using  a  3.5  percent 
escalation . 


CATEGORY 

PROPOSED 

NEGOTIATED 

TOTAL 

RATE 

RATE 

HOURS 

TECHNICAL 

SPECIALIST 

S  24.66 

S  24.42 

10,210 

TECHNICAL 

STAFF 

S  30.64 

S  30.35 

6,512 

TECHNICAL 

SENIOR 

$  21.45 

S  21.24 

10,460 

TECHNICAL 

ENGINEER 

S  17.91 

S  17.74 

9,559 

TECHNICAL 

SUPPORT 

S  13.60 

S  13.47 

812 

LABORATORY 

'  SERVICES 

S  17.27 

S  17.11 

2,006 

TECHNICAL 

PUBS. 

S  15.94 

S  15.78 

4  93 
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Fee  Negotiation  -  LMSC  requested  a  fee  of  9 
percent  of  costs,  exclusive  of  cost-of-monev. 
The  government’s  fee  objective  was  4.9  percent. 


The  government’s  fee  objective  was  4.9  percent 
A  fee  of  5.8  percent,  exclusive  of  cost-of 
money,  was  negotiated. 


Summary  Of  Estimated  Costs  For 
The  Second  Contract  Award 


following  chart  provides 


final 


estimated  costs  for 


second  contract  award  after 


audit  and  negotiation: 

PRICE  BREAKDOWN 


DESCRIPTION 


TOTAL 


Labor  Hours 


40,052 


Labor  Dollars 


Overhead 


G  &  A 


Material 


$  891,783 

1 ,302,025 
361 ,712 
583 ,236 


Purchased  Services 


6 , 000 


Subcontracts 


102 . 968 


Procurement  Burden 


Travel 


22 , 150 
31 .657 


Subtotal 


Cost  of  Money 


S3 , 301 ,531 
91 , 089 


Total  Cost 


S3 , 392 , 620 


192 ,843 


CPFF 


S3 , 585 ,463 


G.  ACTUAL  COST  DATA 


The  level  of  funding,  actual  expenditure  level  and 
planned  spending  as  of  June  30,  1987  is  illustrated 
graphically  on  the  following  page: 
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Actual  cost  data  were  obtained  from  Mr.  !)• 
Dorset!,  LMSC  ’  s  contract  administrator  for  the  AMi, 
project.  The  source  of  the  following  spending  da* a 


the  Weekly 
September  31 
of  the  f 1 r  s t 


Project  Financial  Status  Report 
1'tHb,  which  reflet* s  spending  at 
contract  phase . 

At.Tt'Al.  COSTS 
FIRST  CONTRACT  PHASF 


da 


LABOR  HOURS 
LABOR  COSTS 
0  VFRiiLAD 
0  •/.  A  FXPFNSFS 
OVERTIME  PREMIUM 
MATERIAL 

PURCHASED  SERVICES 

subcontract 

PROCUREMENT  BURDEN 
TRAV FT 


EXPENDED  TO  DATE 
IV  ,  4  2 4  hours 
S  36  3 , 333 
a  ri  6  ,  S  5  H 
1 H4  ,  3  3  3 
3  4 
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CRAN'D  TOTAL 


S  1  .  1  I)  .  a  1  a 


The  difference  between  the  amount  funded  under 
the  contract  and  the  budgeted  total  shown  above  Is  the 
cost  of  money  and  fixed  fee  amounts. 

TOTAL  FUNDED  ($1,350,000)  -  TOTAL  BUDGETED  ($1,243,205) 
«•  COST  OF  MONEY  ($38,450)  +  FIXED  FEE  ($68,345  ). 

Chapter  IV  discusses  the  actual  "budgeted" 


accomplishment  and  measure  of  technological  progress 
for  the  first  and  second  phases  of  the  contract. 

The  following  cumulative  actual  costs  as  of 
September  31,  1987,  provide  actual  expenditures  for  the 

first  and  second  contract  phases. 

ACTUAL  COST 

FIRST  AND  SECOND  PHASES 


EXPENDED  TO  DATE 

BUDGET 

LABOR  HOURS 

55 

,585  hours 

73 , 653 

LABOR  COSTS 

S 

1 .170 ,014 

$ 

1 . 369 ,867 

OVERHEAD 

$ 

1 ,806 ,416 

$ 

2 , 023,831 

G  &  A  EXPENSES 

$ 

542 , 392 

$ 

705 , 570 

OVERTIME  PREMIUM 

$ 

1  18 

$ 

100 

MATERIAL 

$ 

360 , 671 

$ 

1 96 ,203 

PURCHASED  SERVICES 

$ 

59,110 

$ 

61,100 

SUBCONTRACT 

s 

0 

$ 

146 ,700 

PROCUREMENT  BURDEN 

$ 

13,116 

$ 

12 . 380 

TRAVEL 

$ 

25 . 606 

$ 

18 , 064 

OTHER 

$ 

325 

$ 

125 

GRAND  TOTAL 

$ 

3 , 977 , 768 

$ 

4 , 533 ,950 

The  difference 

between 

the  amount 

funded  under 

the  first  and  second 

contract  phases  and 

the 

budgeted 

total  shown  above  is 

the  cost 

of  money 

and 

fixed  fee 

amoun  t  s . 


TOTAL  FUNDED  ($4,935,463) 
-  COST  OF  MONEY  ($146,325 


-  TOTAL  BUDGETED 
)  +  FIXED  FEE  ( $2b 


(  $4.533 ,950  ) 

1,188). 


H  . 


INFORMATION  FROM  THE  GOVERNMENT'S  PROGRAM 

discussion  is  based  on  a 
LTCOL  G.  Lasche,  USA. 
September  3,  1987.  LMSC 


MANAGER 
telephone 
the  ANGAS 
initial  1 y 


The  foil ow 1 ng 
eonversat i on  with 
program  manager  on 
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proposed  the  ANGAS  project  to  the  government.  At  the 
time,  Lockheed  had  recently  completed  the  four  million 
dollar  Winkler  project.  Winkler  was  a  scaled  down 
aircraft-borne  spectrometer  that  utilized  many  of  the 
advanced  technologies  that  were  to  be  used  in  the  ANGAS 
proj  ect . 

Winkler  was  successfully  developed,  tested  and 
flown  at  the  estimated  costs.  LMSC ’ s  proposed  costs 
for  ANGAS  were  evaluated  using  Winkler’s  historical 
cost  data. 

ANGAS’  technology  was  more  advanced  than 
Winkler’s  because  it  is  designed  to  function  in  space 
rather  than  the  earth’s  atmosphere.  According  to  the 
program  manager,  projects  like  ANGAS  that  incrementally 
advance  the  SOA  are  usually  completed  much  closer  to 
the  scheduled  time  and  budgeted  costs  than  projects 
that  attempt  quantum  leaps. 

The  incremental  funding  of  the  ANGAS  contract 
allows  a  structured  development  and  technological 
evaluation  at  key  development  steps.  LTCOL .  Lasche , 
USA,  stated  the  contract’s  SOW  items  and  required  tests 
stipulated  in  each  contract  phase  provide  the  primary 
evaluation  criteria  to  measure  technological  progress. 
To  date  all  SOW  items  and  tests  have  been 
satisfactorily  completed  on  time. 

It  was  also  noted  that  labor  costs  are  usually 
the  most  significant  expense  during  the  research  and 
development  phases  of  an  SOA  extension  project. 


IV.  DATA  ANALYSIS 


A.  INTRODUCTION 

This  Chapter  analyzes  the  data  presented  in  Chapter 
III.  The  analysis  examines  cost  variances,  schedule 
variances  and  the  measurement  of  technological 
progress.  Significant  characteristics  of  Lockheed’s 
cost  estimating  methods  and  models  that  are  potentially 
useful  in  aiding  better  estimation  of  SOA  extensions  by 
the  government  will  also  be  discussed.  The 
contributions  and  role  of  the  government’s  program 
manager  for  the  ANGAS  will  also  be  highlighted  when 
applicable . 

B.  COST  VARIANCE  ANALYSIS 

Table  1  provides  the  dollar  value  and  the 
percentage  of  cost  variances  by  cost  element  for  the 
first  funding  period  of  the  ANGAS  project  as  of  31 
September  1986: 


TABLE  1 

COST  VARIANCES 

COST  ACTUAL  BUDGETED  COST  VARIANCE  AS 

ELEMENT  COSTS  COSTS  VARIANCE  *  OF  BUDGET 


LABOR  $ 
OVERHEAD 
G  &  A 
OVERTIME 
MATERIAL 
PUR.  SERV. 
SUBCONTRACT 
PRO . BURD . 
TRAVEL 


$  369,323 
556,858 
184,323 
24 

15,533 
.  36,099 

CT  0 

1,678 
6,657 


S  302,464 
510,392 
147,780 
0 

110 ,793 
0 

146,700 
9 ,012 
16 , 064 


$  -66,859 
-46,466 
-36 , 543 
24 

+  95 , 260 
+  36 , 099 
+146,700 


22.10  % 
9 . 10 
24 . 73 
NA 
85 . 98 
NA 
109 
81 . 38 
58.55 


Table  1  Indicates  that  negative  cost  variances5 
during  the  first  funding  phase  of  the  ANGAS  contract 
were  realized  In  labor  costs,  overhead,  G  &  A  expenses 
and  overtime  premiums.  These  cost  elements  are 
Influenced  by  labor  costs  either  directly  or  through  an 
applied  rate  >  a  direct  labor  base. 

LMSC  had  budgeted  for  13,900  labor  hours.  The 
number  of  labor  hours  actually  expended  during  the 
first  phase  of  the  contract  was  17,424  hours.  Dr. 
George  Nakano ,  LMSC ’ s  Principle  Investigator  for  the 
ANGAS  program,  explained  that  during  the  early  stages 
of  development  additional  research  and  development 
effort  was  put  Into  the  sol id-cryogen  cooler  system. 
This  was  a  critical  item  in  the  overall  SOA  extension 
effort  that  has  a  long  lead-time  and  would  be  built 
outside  the  Space  Sciences  Laboratory  Division. 

All  other  cost  elements  experience  positive  cost 
variances  during  the  initial  funding.  A  cumulative 
positive  cost  variance  of  $72,690,  5.8  <f>  of  budgeted 
total  costs  was  reported  at  the  conclusion  of  the  first 
contract  period. 

Budgeted  labor  costs,  overhead,  G  &  A  expenses  and 
overtime  premiums,  all  based  on  direct  labor  costs, 
were  budgeted  at  77.27  %  of  total  budget  costs.  Actual 
costs  for  these  cost  elements  accounted  for  94.87  t  of 
actual  total  costs. 

Table  2  provides  the  cumulative  and  percentage 
variances  for  the  first  and  second  contract  phases  as 
of  31  September  1987: 


4  Negative  cost  variances  result  when  actual  costs 
exceed  budgeted  costs. 


TABLE  2 

CUMULATIVE  COST  VARIANCES 


COST 

ACTUAL 

BUDGETED  COST  VARIANCE  AS 

ELEMENT 

COSTS 

COSTS  VARIANCE  %  OF 

BUDGET 

LABOR 

$1,170,014 

$1  ,  369 ,867 

+  $199 ,853 

14 . 59^ 

OVERHEAD 

1 , 806 ,416 

$2 ,023,831 

+  217 .415 

10.75 

G  &  A 

542 , 392 

705  ,  570 

+163 , 178 

23.13 

OVERTIME 

118 

100 

18 

18 

MATERIAL 

360 . 67 1 

196 ,203 

-  1  b  4 ,468 

83 . 82 

PUR . SER . 

59,110 

61,110 

+  2,000 

3 . 2 

SUBCONT . 

0 

146 ,700 

+  146 .700 

100 

PRO . BURD 

13.116 

12 , 380 

736 

5 . 9 

TRAVEL 

25 ,606 

18 , 064 

-  7,542 

41.75 

OTHER 

325 

125 

200 

160 

TOTAL 

$3 , 977 . 768 

S4 ,533 ,950 

+566,1 82 

12 . 2bi 

Table  2  Indicates  that  negative  cost  variances 
existed  at  the  end  of  the  second  funding  phase  in 
materials,  procurement  burden,  travel  and  other,  cost 
element  catagories.  A  cumulative  positive  cost 
variance  of  $556,182,  12.263  percent  of  budgeted  total 
costs,  was  reported  at  the  conclusion  of  the  first  two 
contract  periods. 

LMSC  had  budgeted  for  73,653  labor  hours  at  this 
state  of  the  development  effort.  The  cumulative  number 
of  labor  hours  actually  expended  during  the  first  two 
contract  phases  was  55,585  hours. 

Direct  labor  and  direct  labor-based  costs  were 
budgeted  at  90.4  %  of  total  budgeted  costs.  Actual 
labor-related  costs  were  88.4  of  actual  total  costs. 

Total  actual  costs  for  the  ANGAS  project  have  been 
slightly  less  than  budgeted.  Dr.  Nakano  and  Mr. 
Dorsett,  of  LMSC,  explained  that  the  slightly  slower 
overall  spending  rate  was  partially  attributed  to  the 
uncertainties  and  delays  experienced  in  obtaining 
timely  approval  of  incremental  funding  requests. 
Material  procurement  and  research  efforts  that  are 
needed  for  SOW  items  defined  in  future  phases  of  the 
project  would  have  been  initiated  in  the  first  and 
second  phases  of  the  project  if  future  funding  had  been 
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approved.  The  incremental  funding  approach  adopted  by 
the  government  provides  control  over  the  project  but 
also  makes  the  development  more  difficult  for  the 
cont  ractor  . 

Incentives  and  definite  evaluation  periods  for  a 
level  of  effort  contract  like  the  ANGAS  project  are 
established  through  the  incremental  funding, 
establishment  of  SOW  goals  for  each  contract  phase  and 
the  ability  to  adjust  future  funding  based  on  an 
evaluation  of  past  performance  and  future  potential. 

The  contractor  has  been  provided  incentive  to 
satisfactorily  achieve  stated  SOW  goals  for  the  phase 
of  the  contract  for  which  funding  has  been  reviewed. 

Future  long  lead-time  items  required  in  later 

contract  phases  of  the  ANGAS  project  cannot  be 

initiated  until  funding  of  those  contract  phases  is 

approved . 

C.  SCHEDULE  VARIANCE  AND  TECHNOLOGICAL  PROGRESS 

Schedule  variances  provide  a  way  to  highlight 

situations  where  effort  is  behind  or  ahead  of  schedule. 

The  traditional  methodology  for  analyzing  schedule 

variances  compares  the  budgeted  cost  of  work  performed 

and  the  budgeted  cost  of  work  scheduled.  In  a  level  of 

effort  development  contract,  such  as  the  ANGAS  project, 

a  determination  of  the  budgeted  cost  of  work  performed 

becomes  almost  impossible  to  calculate  due  to  the 

intangible  nature  of  the  goals  and  SOW  items.  An 

evaluation  of  technical  progress  is  a  more  useful 

measure  when  used  with  the  project’s  cost  variances  to 

evaluate  overall  benefits  and  project  status. 

Analysis  of  technical  variance  is  probably  the  most 
crucial  type  of  analysis  involved  in  the  management 
process.  It  is  the  one  where  assistance  of  other 
technical  experts  may  be  required.  It  may  also 

require  some  restructuring  of  the  program  merely  to 
determine  the  cause  of  the  variance.  Note  that  any 


significant  technical  variance  will  always  result  in 
cost  and  schedule  variances.  LREF.  2:p.  4-20J 

Since  no  significant  cost  or  schedule  variances 
have  occurred  in  the  ANGAS  project,  the  most  obvious 
indicators  of  unsatisfactory  technical  progress  are  not 
evident.  LMSC  has  satisfactorily  completed  all  SOW 
items  listed  in  the  first  and  second  phases  of  the 
contract.  To  further  analyze  technical  progress,  the 
ANGAS  project’s  system  engineering  will  be  examined. 

MIL-STD  449  defines  system  engineering  as: 

the  application  of  scientific  and  engineering  efforts 
to:  transform  an  operational  need  into  a  description 
of  system  performance  parameters  and  a  system 
configuration  through  the  use  of  an  incentive  process 
of  definitions,  synthesis,  analysis,  design,  test  and 
evaluation;  integrate  related  technical  parameters 
and  assure  compatibility  of  all  physical,  functional 
and  program  interfaces  in  a  manner  which  optimizes 
the  total  system  definition  and  design,;  integrate 
reliability,  maintainability.  safety,  survivability, 
human  and  other  such  factors  Into  the  total 
engineering  effort. 

The  contract  SOW  specifies  the  minimum  technical 
requirements  in  the  contract.  Appendix  B  contains 
copies  of  the  first  and  second  contract  phase 
Statements  of  Work.  Examination  of  these 
specifications  reveals  that  they  are  composed  of 
specific  configuration,  design,  performance  and 
reliability  and  maintainability  requirements.  The 
program  manager,  LTCOL  G.  Lasche ,  USA,  stressed  that 
the  lessons  learned  in  the  development  of  the  Winkler 
project,  which  used  similar  technology,  was  a  basis  for 
the  technical  requirements  developed  for  ANGAS.  ANGAS 
system  engineering  appears  to  be  a  detailed 
specification  of  valid  technical  goals  developed  from 
past  experience  and  an  overall  knowledge  of  the 
technologies  involved. 

It  is  logical  that  any  analysis  of  technological 
progress  be  based  on  status  reviews  of  system 
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engineering  goals.  The  government’s  acquisition 

process  requires  formal  and  informal  design  reviews  to: 

determine  the  adequacy  of  contractor  and  DOD  in- 
house  efforts  towards  achieving  design  goals. 
Participants  should  include  design  attribute 
specialists  in  reliability,  maintainability,  safety, 
and,  particularly,  logistic  suppor tab i 1 i ty .  Reviews 
should  include  a  preliminary  design  review,  a 
critical  design  review  (CDR  )  ,  a  design  certification 
review,  a  functional  configuration  audit,  a  physical 
configuration  audit,  and  a  first-article 

configuration  inspection.  [REF.  2:p.  4-49] 

The  most  recent  formal  review  of  the  ANGAS  project 

was  the  CDR  which  was  presented  to  DARPA  on  July  21  and 

22  of  1987. 

The  CDR,  a  formal  review  of  the  detailed  design  of  a 
configuration  item,  is  performed  by  the  program 
manager  late  in  the  prototype  subphase  when  the 
design  detail  is  essentially  complete,  but  prior  to 
drawing-release  and  fabrication  of  formal  test 
articles.  [REF.  2:p.  4-49J 

The  critical  design  review  demonstrated  that 

technical  and  system  engineering  goals  for  the  ANGAS 

project  were  accomplished  satisfactorily.  The  CDR 

specifically  addressed: 

1.  System  Engineering 

2.  Germanium  Detector  Subsystem  Development  (a 
critical  technical  component  of  the  ANGAS 
system) 

3.  Shield  Design  (a  critical  technical  component 
of  the  ANGAS  system) 

4.  Mechanical  Engineering  specifications 

5.  Electrical  Subsystem  specifications 

6.  Source  Deployment  Subsystems 

7.  Cooler  Subsystem  specifications  (a  critical 
technical  component  of  the  ANGAS  system) 

8.  Electrical  Ground  Support  Equipment 

specifications 

9.  Monitor  Subsystem  specifications 

10.  Instrument  Test  Plan 

11.  Reliability,  Quality  and  Safety 

12.  Instrument  Development 

In  summary,  ANGAS  technical  goals  and  objectives 

have  been  accurately  detailed  and  specified  as  contract 


requirements  based  on  past  experience  obtained  from  the 
development  of  systems  using  similar  technologies. 
Formal  and  informal  reviews  and  evaluations  of 
technical  requirements  have  proved  that  the  ANGAS 
project’s  technical  goals  are  both  viable  and  capable 
of  being  accomplished. 

This  research  effort  does  not  attempt  to  define  a 
quantitative  measurement  technique  for  evaluating 
technical  variance  and  accomplishment.  An 

understanding  of  the  long-range  R  &  D  planning  steps 
developed  by  Ward  D.  Lowe  and  presented  in  Chapter  III 
is  a  useful  guide  in  establishing  initial  technical 
objectives  for  a  proposed  SOA  extension  project. 

The  prior  development  of  the  Winkler  Program 

reduced  the  risks  and  development  and  provided 

technical  information  on  some  of  the  new  technologies 
used  in  ANGAS.  The  following  brief  description  of  the 
relatively  simple  mul ti -attribute  utility  method, 
summarized  for  the  Navy  Program  Manager’s  Guide, 

provides  an  avenue  for  future  study  of  quantification 
of  technical  goals  and  measurement. 

The  model  addresses  the  management  of  risk  that  is 
inherent  in  decision  making  with  incomplete 
information.  Establishing  SOA  extension  technological 
objectives  and  measurement  specifications  at  the 
initiation  of  a  project  is  often  based  on  Incomplete 
Information.  The  use  of  this  limited  information  can 
be  optimized  through  a  five  step  process  which  consists 
of : 

1.  Breaking  down  the  tasks  to  be  accomplished  into 
manageable  components  or  attributes. 

2.  Estimating  the  utility  factor,  the  relative 
importance  of  each  component  or  attribute. 

3.  Developing  a  utility  function  or  curve  which 
describes  the  utility  values  as  a  function  of 
some  descriptive  variable  (i.e.,  reliability  in 
terms  of  mean  time  between  failure.  ) 
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4.  Estimating  the  risks  associated  with  attaining 
the  utility  values  chosen  for  each  attribute. 

5.  Developing  options  to  avoid  or  overcome 
obstacles  to  success  and  to  compare  alternative 
paths,  solutions,  or  concepts. 


D.  SIGNIFICANT  CHARACTERISTICS  OF  LOCKHEED’S  COST 

ESTIMATING  METHODS  AND  MODELS  THAT  CAN  AID 

GOVERNMENT  COST  ESTIMATING 

Lockheed  sometimes  uses  bottom-up  cost  estimating 
which  is  usually  based  on  historical  data  and  provides 
a  high  degree  of  accuracy.  Cost  estimating  and 
evaluation  within  DOD  should  utilize  historical  costs, 
for  completed  projects  that  are  similar  in  scale  and 
technology,  to  evaluate  projects  under  consideration. 

Use  of  parametric  cost  models,  specifically  the  RCA 
PRICE  Hardware  model  is  used  by  Lockheed  during  the 
early  development  and  concept  exploration  phases  when 
historical  cost  data  does  not  exist.  Parametric  cost 
models,  which  allows  economic  SOA  extension  trade-off 
and  schedule  analysis,  should  be  used  by  DOD  when 
historical  data  does  not  exist  early  in  development 
stages . 

The  Lockheed  Internally  developed  STAR  model 
combines  the  characteristics  of  a  simi lar-to-model  and 
an  associated  model.  This  model,  which  was  described 
in  detail  in  Chapter  II,  is  based  on  a  historical 
database  that  has  been  developed  over  a  20  year  period. 
The  data  base  represents  actual  production  and 
development  cost  experience,  which  can  be  retrieved  by 
line  item  or  box  level  based  on  relatively  simple  input 
characteristics.  Development  of  similar  integrated 
data  base  and  models  within  DOD  or  major  components 
would  significantly  enhance  the  government’s  ability  to 
accurately  estimate  SOA  extension  costs. 
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V.  CONCLUSIONS 

The  conclusions  drawn  from  this  research  effort  are 
based  on  the  data  and  analysis  of  the  ANGAS  project. 
These  conclusions  can  be  used  as  a  basis  for  future 
studies  involving  SOA  extension  cost  estimating  methods 
and  case  studies. 

Primary  Research  Question:  How  do  estimated 

development  and  production  costs  compare  with 
actual  costs  for  the  ANGAS  project? 

Development  costs  were  the  primary  costs  estimated 
and  recorded  to  date  due  to  the  early  stage  of 
development  of  the  ANGAS  project.  Total  actual  costs 
were  slightly  lower  than  total  budgeted  costs.  Direct 
labor  and  labor  related  costs  were  higher  than 
budgeted  during  the  first  phase  of  development. 

Variances  did  exist  within  Individual  cost  elements.  No 
cost  element  experienced  negative  cost  variances  during 
both  contract  phases  that  were  evaluated  with  the 
exception  of  a  S24  and  $18  cost  overrun  in  overtime 
premiums . 

Subsidiary  Research  Questions; 

1 .  What  methods  and  techniques  were  used  to  estimate 
development  costs? 

LMSC  used  a  bottom-up,  engineering  cost  estimating 
methodology  to  estimate  development  costs.  These 
estimates  were  generated  within  the  Lockheed  Space 
Sciences  Laboratory  based  on  past  experience  of  costs 
Incurred  developing  similar  satellite  borne  systems. 
The  bottom-up  estimating  methodology  used  at  LMSC 
provided  detailed  functional  and  cost  element  estimates 
based  on  historical  cost  data  and  specific  contractual 
requirements  as  detailed  in  the  WBS ,  SOW  and  RFP. 
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2.  How  was  technological  progress  measured  and 

evaluated? 

Technological  progress  was  measured  and  evaluated 
by  monitoring  completion  of  the  specific  configuration, 
design,  performance  and  reliability  and  maintainability 
requirements  stated  in  the  ANGAS  Statements  of  Work 
during  the  formal  and  informal  project  reviews.  The 
program  manager  for  AXGAS  was  able  to  state  technical 
goals  and  objectives  based  on  experience  gained  from 
the  development  of  a  similar  spectrometer  system  that 
used  many  of  the  same  advanced  technologies. 

3.  What  factors  helped  or  hindered  accurate  cost 
estimating  and  accomplishment  of  stated  SOA 
extension  goals? 

Accurate  cost  estimating  was  enhanced  by  the 
existence  of  historical  data  on  similar  type  SOA 
extension  development  costs.  The  division  of  LMSC 
that  was  developing  the  ANGAS  project  was  also 
experienced  in  developing  similar  systems.  ANGAS 
incrementally  extended  the  SOA  based  on  a  smaller  scale 
project  that  combined  many  of  the  same  technologies  but 
was  designed  to  function  in  the  earth’s  atmosphere 
instead  of  outer  space.  Historical  cost  data,  technical 
experience  and  incremental  SOA  extension  enhanced  cost 
estimating  accuracy  and  accomplishment  of  technological 
goals . 

Factors  that  hinder  accurate  cost  estimating  and 
accomplishment  of  stated  SOA  extension  goals  arise  from 
the  inherent  risks  of  decision  making  with  incomplete 
information.  The  inability  for  the  government  to 
accurately  specify  project  design  and  specification 
requirements  and  monitor  accomplishment  through  formal 
and  informal  reviews  greatly  hinders  accomplishment  and 
measurement  of  stated  SOA  extension  goals.  Lack  of 
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historical  data,  technical  experience  and  inability  to 
correctly  interpret  contract  requirements  hinders 
contractors  from  correctly  estimating  SOA  extension 
costs . 

Findings ;  There  were  no  significant  variances 
between  total  estimated  and  total  actual  costs  to  date. 
Due  to  the  contractor  uncertainties  associated  with  the 
incremental  funding  used  in  the  project  the  spending 
rate  was  slightly  lower  than  planned. 

Incremental  funding  of  a  DOD  SOA  extension  project 
definite  contract  phases  and  the  opportunity  to  specify 
and  evaluate  technological  objectives  during  each 
phase.  This  type  of  funding  provides  Incentives  for  the 
contractor  to  successfully  achieve  technological 
objectives  to  receive  funding  for  later  phases. 
Incremental  funding  provides  leverage  and  flexibility 
to  the  government  in  reducing  cost  overruns  and 
achieving  technological  goals.  The  length  of  time 
required  for  the  development  effort  is  increased  with 
incremental  funding. 

Direct  labor  costs  and  costs  applied  as  a 
percentage  of  direct  labor  costs  were  the  most 
significant  costs  during  the  development  phases.  These 
costs  accounted  for  over  75  percent  of  total  costs  in 
each  phase. 

SOA  extensions  that  incrementally  advance  the  SOA 
reduce  the  risk  of  costs  overruns  and  failure  to 
achieve  technological  goals.  The  historical  cost  data 
and  expertise  obtained  through  the  development  of  the 
smaller  scale  Winkler  project  enabled  DARPA  to 
accurately  detail  system  engineering  specifications  and 
technical  test  parameters. 

The  measurement  and  analysis  of  cost  variances  and 
technological  progress  is  the  primary  gauge  o  f 


effectiveness  for  a  research  and  development  effort 
administered  by  a  level  of  effort  contract .Technii a! 
progress  for  the  A.VGAS  project  was  evaluated  as 
satisfactory. 

Currently  available  parametric,  analogous  and 
similar--to  cost  estimating  models  used  by  LMSC  and 
private  industry  could  be  adopted  to  a  greater  degree 
by  the  DOD  to  enhance  SOA  cost  estimating  accuracy. 

Recommendations  for  Further  Study :  Additional  case 
studies  are  needed  *■  o  Identify  trends  in  the  cost 
estimating  techniques  and  accuracy  of  SOA  extension 
efforts.  The  computer  based  cost  estimating  models  that 
are  discussed  in  this  research  effort  and  many  other 
computer  based  models  not  addressed  in  this  project 
merit  a  great  deal  of  additional  study.  Identifying 
cost  estimating  methods  that  will  allow  the  most 
efficient  use  of  limited  DOD  funding  by  cost 
effectively  including  new  technologies  in  SOA 
extensions  and  new  system  development  are  vital 
national  objectives. 


APPENDIX  A 
DEFINITIONS 


Bottom-Up  Cost  Estimation  Model:  A  Cost  estimating 
procedure  based  on  detailed  functional  and  cost 
element  estimates  prepared  at  the  lowest  practical 
level  of  task  and  design  detail. 

Budget:  Estimated  costs  of  the  effort  defined  in  the 

work  packages  and  level  of  effort  SOW  Items.  These 
costs  should  be  broken  out  at  the  cost  account 
level.  The  estimated  cost  of  the  effort  scheduled 
to  be  accomplished  at  a  point  In  time  within  DQD  Is 
termed  "Budgeted  Cost  of  Work  Scheduled"  (BCWS). 


Critical  Design  Review  ( CDR  )  :  A  formal  review  of  the 

detailed  design  of  a  configuration  Item  is 
performed  by  the  program  manager  late  In  the 
prototype  subphase  when  the  design  detail  is 
essentially  complete  but  prior  to  drawing  release 
and  fabrication  of  formal  test  articles. 


Development  Costs:  Costs  associated  with  research, 

planning  and  design  for  a  new  product  or  process  or 
for  a  significant  Improvement  to  an  existing 
product  or  process.  Includes  the  conceptual 
formulation.  design.  and  testing  of  product 
alternatives  and  construction  of  prototypes. 


Intra-Lockheed  Work  Transfer  ( IWT  )  Items:  Material 
Items  that  are  made  at  Lockheed  and  transferred  at 
cost  to  another  entity  within  the  company. 
Includes  a  separate  breakdown  by  cost. 


Parametric  Modeling:  Cost  estimating  procedure  which 

utilizes  a  model  comprised  of  algorithms  and 
mathematic  equations.  Estimated  costs  are  derived 
from  relationships  of  cost  to  physical  or 
performance  characteristics. 


Purchased  Items:  Includes  material  Items  not 

considered  standard  commercial  Items  or  raw 
materials. 


Raw  Materials:  Materials  in  a  form  or  state  that 

require  further  processing. 


Standard  Commercial  Items:  Consists  of  Items  that  the 
contractor  normally  fabricates,  in  whole  or  in 
part,  and  that  are  generally  stocked  Inventory. 


State  of  the  Art  (SO A)  Extensions:  Used  In  this  thesis 

to  describe  a  new  weapon  system  or  component  that 
has  incorporated  an  advancement  In  technology  in  a 
unique  process  or  for  a  unique  application. 


WBS  Dictionary:  The  WBS  dictionary  defines  each  WBS 

element  and  describes  the  techn i cal / f unc t i onal  task 
content  and  responsibilities. 

Work  Break  Down  Structure:  A  produc t -or i ented  family 
tree  composed  of  hardware,  services  and  data  which 
results  from  project  engineering  effort  during  the 
development  and  production  of  a  defense  material 
item  and  which  completely  defines  the  project  7 
program . 


APPENDIX  B 


STATEMENT  OF  WORK 
FIRST  CONTRACT 

1.  The  work  and  services  to  be  performed  hereunder 
shall  be  subject  to  the  requirements  and  standards 
contained  In  Exhibit  A  and  the  following  paragraph ( s ) . 

2.  The  Contractor  shall  carry  out  a  program  of 
research  in  advanced  gamma-rav  detectors  for  use  in 
space  which  shall  include  the  following  tasks: 

1.  Conduct  gamma-ray  imaging  studies, 

2.  Establish  instrument  GSE  requirements, 

3.  Conduct  preliminary  design  of  germanium  sensor 
and  anticoincidence  shield, 

4.  Prepare  digital  signal  processing  system 
requirements  and  specifications, 

5.  Prepare  cryogenic  cooling  system 

specifications, 

6.  Conduct  preliminary  design  of  collimator, 
wheel,  onboard  hardware.  germanium  sensor 
electronics,  and  antishield  housing  and 
electronics , 

7.  Prepare  germanium  sensor  procurement 

specifications  and  antishield  crystal 

procurement  specifications, 

8.  Complete  final  cooler  design  and  performance 
simulation . 


STATEMENT  OF  WORK 
SECOND  CONTRACT  PHASE 


Under  Modification  P00001,  the  Contractor  shall 
conduct  the  final  design  and  sensor/shield  evaluation 
of  a  space  demonstration  experiment  known  as  the  DARPA- 
201  ANGAS  Experiment  aboard  a  free-flyer  satellite 
mission.  The  work  shall  include  the  following  tasks: 

(1)  Finalize  design  of  a  high  sensitivity, 
fine  energy  resolution  imaging  spectrometer,  known  as 
the  ANGAS  instrument,  background  monitors  and  auxiliary 
equ i pmen  t . 

(2)  Perform  laboratory  evaluations  and  bench 
tests  of  the  Ge  sensor  design  and  of  the  Nal  shield 
system. 

(3)  Conduct  design  studies,  computer 
simulations  and  laboratory  bench  tests  for  performance 
verification  of  the  imaging  techniques  to  be  employed 
on  the  ANGAS  and  finalize  the  design  of  the  collimator 
mask  system  for  the  flight  instrument. 

(4)  Provide  the  electrical,  thermal,  and 
environmental  specifications  and  other  interface 
information  necessary  to  integrate  the  DARPA-201 
payload  aboard  the  satellite. 

(5)  Provide  all  physical,  procedural, 
functional,  and  safety  data  necessary  to  generate  an 
Experiment  Requirements  Document  (ERD)  consisting  of: 

(a  )  Interface  Control  Document. 

(b)  Ground  Operations  Requirements 
Document . 

(c)  Flight  Operations  Requirements 
Document. 

(6)  Provide  the  inputs  necessary  to  meet  Un¬ 
identified  DARPA-201  milestone  schedules.  Departures 
from  these  schedules  must  be  mutually  agreed  to  by 
DARPA  and  the  LPARL . 

(7)  Maintain  an  experiment  quality  program 
during  the  design  and  development  of  the  payload. 

(8)  Design  and  initiate  development  of  the 
ground  support  equipment  ( GSE  )  necessary  to  test  and 
integrate  tne  DARPA-201  payload  with  the  free-flyer 
satel 1 i te . 

(9)  Provide  DARPA-201  progress  and  status 
information  and  support  the  mission  working  group  and 
status  reviews  as  necessary. 


STATEMENT  OF  WORK 
PROPOSED  16.5  MILLION  DOLLAR 
COMPLETION  CONTRACT 


The  Lockheed  Missiles  and  Space  Company  will 
provide  to  the  Defense  Advanced  Research  Agency  and  the 
Office  of  Naval  Research  the  necessary  personnel, 
equipment  facilities  and  services  to  conduct  a  space 
demonstration  experiment  known  as  the  DARPA-201  AN'GAS 
Experiment  aboard  a  free-flyer  satellite  mission.  In 
the  conduct  of  this  work  the  following  tasks  will  be 
perf o  rmed : 

1  )  Design.  develop,  test  and  calibrate  a  high 
sensitivity,  fine  energy  resolution  imaging 
spectrometer,  known  as  the  ANGAS  instrument  and 
an  auxiliary  background  monitor. 

2)  Conduct  design  studies,  computer  simulations 
and  bench  tests  for  performance  evaluation  of 
the  Imaging  techniques  for  intended  employment 
on  the  ANGAS  collimator  mask  wheel. 

3)  Based  on  the  timely  receipt  of  the  electrical, 
thermal  and  environmental  specifications  and 
other  interface  information  from  the 
Government,  provide  the  necessary  Information 
to  integrate  the  DARPA-201  payload  aboard  the 
satellite. 

4)  Provide  all  physical,  procedural,  functional 

and  safety  data  necessary  to  generate  an  Experiment 
Requirements  Document  (ERD)  consistent  with  the  vehicle 
and  Interface  requirements  specified  In  the  STP  From 
1721  (1085).  The  ERD  consists  of 

a)  Interface  Control  Document 

b)  Ground  Operations  Requirements  Document 

c)  Flight  Operations  Requirements  Document 

5)  Provide  the  Inputs  necessary  to  meet  the 
identified  DARPA-201  milestone  schedules. 

6)  Flight  qualify  the  DARPA-201  payload  through 

the  maintenance  of  an  experiment  quality  program  during 
the  design  and  development  of  the  payload.  The 

Principal  Investigator  will  have  full  responsibility 
for  conducting  all  aspects  of  the  quality  program.  The 
qualify  program  practices  with  the  ONR. 

V)  Provide  the  ground  support  equipment  ( GSK  ' 

necessary  to  test  and  Integrate  the  DARPA-201  payload 
with  the  free-flyer  satellite. 

S  )  Provide  logistics  and  ope  ration  support  to  the 
DARPA-201  payload  during  definition,  integration  .  test 
and  on  -  orb  1 t  operations. 

0)  Provide  DARPA-201  progress  and  status 

informal  i  . . : ,  using.  the  Lockheed  R  ,v  1)  st  alula:  1 


management  and  accounting  practices.  Support  the 
mission  working  group  and  status  reviews  as  necessary. 

10)  Based  on  the  timely  receipt  of  government- 
furnished  computer-compatible  agency  data  tapes, 
analyze  the  first  two  years  of  on-orbit  data. 

11)  Provide  flight  performance  reports  as  required. 

12)  Submit  the  scientific  results  acquired  with  the 
DARPA-201  instruments  for  publication  in  the  open 
technical  literature. 
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MAJOR  MILESTONES  FOR  THE 
DARPA-20 1  ANGAS  EXPERIMENT 

Project 

Milestones  1985  1986  1987  1988  1989  1990  1991  1992 

Harware  Design - 

Procurement - 

I CD  Slgnoff - 

Ground  Operations - 

required  documentation 

Flight  Operations - 

required  documentation 

Hardware  Development - 

Instrument  Test - 

Hardware  Delivery - 

Spacecraft  Prelaunch  Support - 

Launch - 

Post  Launch  Operations - 
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